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Abstract —Given a program and a time deadline, does the 
program finish before the deadline when executed on a given 
platform? With the requirement to produce a test case when such 
a violation can occur, we refer to this problem as the worst-case 
execution-time testing (WCETT) problem. 

In this paper, we present an approach for solving the WCETT 
problem for loop-free programs by timing the execution of a 
program on a small number of carefully calculated inputs. We 
then create a sequence of integer linear programs the solutions 
of which encode the best timing model consistent with the mea¬ 
surements. By solving the programs we can find the worst-case 
input as well as estimate execution time of any other input. Our 
solution is more accurate than previous approaches and, unlikely 
previous work, by Increasing the number of measurements we 
can produce WCETT bounds up to any desired accuracy. 

Timing of a program depends on the properties of the platform 
it executes on. We further show how our approach can be used 
to quantify the timing repeatability of the underlying platform. 

I. Introduction 

Execution-time analysis is central to the design and verifi¬ 
cation of real-time embedded systems. In particular, over the 
last few decades, much work has been done on estimating 
the worst-case execution time (WCET) (see, e.g. Q, El, 0). 
Most of the work on this topic has centered on techniques 
for finding upper and lower bounds on the execution time 
of programs on particular platforms. Execution time analysis 
is a challenging problem due to the interaction between 
the interaction of a large space of program paths with the 
complexity of underlying platform (see, e.g., HI, 0). Thus, 
WCET estimates can sometimes be either too pessimistic (due 
to conservative platform modeling) or too optimistic (due to 
unmodeled features of the platform). 

The above challenges for WCET analysis can limit its 
applicability in certain settings. One such problem is to verify, 
given a program P, a target platform H, and a deadline d, 
whether P can violate deadline d when executed on H — 
with the requirement to produce a test case when such a 
violation can occur. We refer to this problem as the worst- 
case execution-time testing (WCETT) problem. 

Tools that compute conservative upper bounds on execution 
time have two limitations in addressing this WCETT problem; 
(i) if the bound is bigger than d, one does not know whether 
the bound is too loose or whether P can really violate d, 
and (ii) such tools typically aggregate states for efficiency 
and hence do not produce counterexamples. Moreover, such 
tools rely on having a fairly detailed timing model of the 
hardware platform (processor, memory hierarchy, etc.). In 
some industrial settings, due to IP issues, hardware details are 
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not readily available, making the task much harder for timing 
analysis (see e.g, this NASA report for more details 0) ; in 
such settings, one needs an approach to timing analysis that 
can work with a “black-box” platform. 

In this paper, we present an approach to systematically test 
a program’s timing behavior on a given hardware platform. 
Our approach can be used to solve the WCETT problem, 
in that it not only predicts the execution times of the worst- 
case (longest) program path, but also produces a suitable test 
case. It can also be adapted to produce the top K longest 
paths for any given K. Additionally, the timing estimate for 
a program path comes with a “guard band” or “error bound” 
characterizing the approximation in the estimate — in all our 
experiments, true value was closed to the estimate and well 
within the guard band. 

Our approach builds upon on prior work on the GameTime 
system Q, ID. GameTime allows one to predict the execution 
time of any program path without running it by measuring a 
small sample of “basis paths” and learning a timing model of 
the platform based on those measurements. 

The advantage of the GameTime approach is that the 
platform timing model is automatically learned from end- 
to-end path measurements, and thus it is easy to apply to 
any platform. However, the accuracy of GameTime’s estimate 
(the guard bands) depend on the knowledge of a platform 
parameter /imax which bounds the cumulative variation in the 
timing of instructions along a program path from a baseline 
value. 

Eor example, a load instruction might take just 1 cycle if the 
value is in a register, but several 10s of cycles if it is in main 
memory and not cached. The parameter /imax can be hard to 
pre-compute based on documentation of the processors ISA 
or even its implementation, if available. 

Our approach shares GameTime’s ease of portability to any 
platform, and like it, it is also suitable for black-box platforms. 
However, rather than depending on knowledge of /tmax, we 
show how one can compute the guard bands using an integer 
linear programming formulation. Experimental results show 
that our approach can be more accurate than the original 
GameTime algorithm 0, at a small extra computational cost. 
Moreover, our algorithm is tunable: depending on the desired 
accuracy specified by a user, the algorithm measures more 
paths and yields more precise estimate; possibly measuring all 
paths if perfect accuracy is requested. Einally, we also show 
how to estimate the parameter /tmax- 


int f(int x) { 

if (X % 2 == 0) { 
if (x & 101) { 

X++; 

} else { 
x+=7; 

} 

} 

return x; 

} 


i 




return x 
I 


Fig. 1: Source code (top) and its corresponding control flow 
graph (bottom) 


II. Preliminaries 

Our solution to the problem is an extension of the 
measurement-based GameTime approach m. We now 
present the model used in im as well as in this paper. 

A. Model 

To decide whether a program can exceed a given time 
limit d, it suffices to decide whether the worst input exceeds 
the limit d. However, recall that without any restrictions on the 
program, when the program contains unbounded loops or re¬ 
cursion, even determining whether the program terminates not 
even the number of steps it performs is undecidable. Therefore, 
we consider only deterministic programs with bounded loops 
and no recursion. We did not find this limitation restricting 
as reactive controllers are already often written with this 
limitation in mind. 

Given a computer program, one can associate with it the 
control-flow graph (CFG) in the usual way (Figure [T}; vertices 
representing locations and edges the basic blocks (straight-line 
code fragments) of the program. Since we assume that the 
loops are bounded and there is no recursion, the loops can 
be unrolled and all function calls inlined. Thus, the resulting 
CFG is always a directed and an acyclic graph (DAG). 

Given a DAG G = {V,E), with the set of vertices V, set 
of edges E, we designate two vertices; source s and sink t 


corresponding to the entry and exit points of the program, 
respectively. 

For vertex v of graph, we use in(ii) to denote the set of 
incoming edges to v and out(u) to denote the set of outgoing 
edges from v. 

To model im the execution times of a program, we 
associate with each edge e G E cost We- The cost Wg models 
the (baseline) execution time of the particular statement the 
edge e corresponds to. 

As described in the introduction, we measure only execution 
times of entire source-to-sink paths and not of individual state¬ 
ments. Given a source-to-sink path x, the baseline execution 
time of the path x is Wx = where the sum is over all 

edges present in x. However, due to caching, branch misses, 
etc. the actual execution time differs from the baseline and thus 
the execution time (the length of the path) can be modeled as 

Wx = y^We + dx 

e^x 

where the term dx denotes the variation from the baseline 
value. The term dx is a function of the input and the context the 
program runs in. This is known as the bounded path-dependent 
repeatability condition ca. 

The length of path x is denoted Wx- Observe that different 
inputs corresponding to the same path in general take different 
time to execute. However, we assume that |(i| < px for some 
p.x G K. We denote the maximum max^^ px by /imax- The 
value Pniax is a measure of timing repeatability. If Pmax = 0 
then the system is perfectly time repeatable. In general, the 
larger the value of the less repeatable the timing of the 

system is. 

The aim of this paper is to find a path x such that Wx 
is maximal. The algorithm in HD as well as ours, does not 
require the knowledge of px’s or pmax to find the worst- 
case execution input. However, the accuracy of the algorithms 
depend of Pmax, as that is inherent to the timing of the 
underlying hardware. 

Our algorithm carefully synthesizes a collection of inputs 
on which it runs the given program and measures the times 
it takes to execute each of them. Then, using these measure¬ 
ments, it estimates the length of the longest path. 

Formally, the pair {x,Wx) consisting of a path x and its 
length Wx is called a measurement. We denote the length of 
the longest path by wm- 

To summarize, throughout the paper we use the following 
notation. 

. G - underlying DAG 

• 5” - set of measured paths 

• Ad = (i)} - set of measurements consisting of pairs 

path Xi G S and the observed length li 

It was shown in HD how, using only \E\ measurements 
of source-to-sink paths, to find an input corresponding to a 
path of length at least wm — 2|i7|pmax- In particular, if the 
longest path is longer than the second longest path by at least 
2\E\pjnsix, the algorithm in ifTTll in fact finds the longest path. 
Thus, we say that the “accuracy” of the algorithm is 2|i7|/iniax- 







Fig. 2: DAG with exponentially many paths 


In this paper, we show how to (i) improve the accuracy without 
increasing the number of measurements, (ii) by increasing the 
number of measurements improve the accuracy even further, 
(iii) how to estimate the timing repeatability of the underlying 
platform (as captured by yUniax)- 

Our algorithm as well as the algorithm in ini measures the 
length of some paths and then estimates the lengths of other 
paths based on those measurements. Note that as long as not 
all the lengths of all the paths are measured, the inaccuracy 
of estimates is unavoidable. Consider for example the graph 
in Figure and assume the graph consists of n “diamonds”. 
Clearly, there are 2" source-to-sink paths in the graph. 

Assume that Wg = 1 for each edge and = 0 for all paths 
X except for one path y for which Hy = /imax > 0. Now, 
suppose we measure the lengths of some collection of paths S. 
As long as S does not contain y, the length of every observed 
path is 2n. Hence, any length of Wy of y in the interval 
[2n — Umax, 2n + Umax] IS Consistent with the measurement. 
Therefore, in the worst case, the best achievable estimate of 
the length of Wy can always be at least p.niax from the real 
answer. 

0 

We now briefly describe the algorithm in ini; we skip the 
standard technical details such as CFG extraction or how to 
And an input corresponding to a given path and focus only on 
how to extract the longest path. 

Let m be the number of edges in E. Then, by numbering 
the edges in E, one can think of each path a; as a vector 
in K’" such that 


J 0 if ith edge is not used in x 
1 if ith edge is used in x 


Now, given two paths x and y, one can define the linear 
combination a • a; + 6 • t/ for a, 6 G M in the natural (component 
wise) way. Thus, one can think of paths as points in an m- 
dimensional vector space over K. In particular, it was show 
in im that there is always a basis B of at most m source-to- 
sink paths such that each source-to-sink path x can be written 
as a linear combination of paths from B: 


Px = ^Cb-pb 
bGB 

where Cb’s are coefficients in K. Moreover, using theory of 2- 
barycentric spanners US, it was shown that B can be chosen 


in such a way that for any path x, it always holds that |c{,| < 2 
for every b G B. The paths in B are called the basis paths as 
they suffice to express any other path. 

Now, if the path can be written as p^ = ■ Pb 

then its estimated (baseline) length is 

Wx = y^^cb-wb 

bGB 

where wt’s are the measured lengths of the basis paths. 

The algorithm thus runs the program on the inputs corre¬ 
sponding to the basis paths in order to measure the lengths 
Pb for each b G B. Moreover, it was shown in mil how, 
by encoding the problem as an integer-linear-program (ILP) 
instance, to find path X such that the corresponding estimated 
length; px = ■ Wb is maximizecH 

Consider the accuracy of the estimated length of px- By 
construction, px = J2beB ^b ■ Pb- Hence, 

e^X beB eeb 

Further, for b G B, we have Wb = J2eeb + ^b- Hence, 


wx -'^Cb • Wb 

— 

y^We+dx-y^Xb-Wb 

bGB 


eGX bGB 


= 

y2 We+dx - y2 

e^X b^B eGh 


= 



e&X b^B e&b 

+dx — ^ Cb ■ db\ 


beB 


< 

< 


dx — ^ Cb ■ db 
bGB 

{2\B\ -f l)/rmax 


Thus, the algorithm in im, finds the longest path (and the 
corresponding input) only up to the error term of (2|i3| + 
l)ftmax, under certain assumptions outlined in ifTTl . A chal¬ 
lenge, as noted earlier, is that it is not easy to estimate the 
value of Pmax- Consider the first four columns in Table |V] 
The second column in the table shows the lengths of the 
longest path as estimated by the algorithm in im. However, 
the third column shows the actual length of the path that is 
measured when the program is executed on the corresponding 
inputs. Notice that in some cases the prediction does not 
match the measured time. Also, the algorithm in ifTH does not 
provide any error bounds on the estimates, thereby making the 
predicted values less useful. 

In this paper we show how, given the exactly same set of 
measurements as in nil, we can find a tighter estimate and 
how to incorporate the knowledge of additional measurements 
to obtain even tighter bounds. In fact, for the benchmarks given 


^In general, however, it holds that the more paths are included in S the 
better is the estimate of the longest path. 


^In case the resulting path is infeasible in the program, a constraint is added 
into the ILP and the ILP is solved again. 










a^o xi 



Fig. 3: Basis paths for DAG in Figure For example, the 
path that always takes the bottom path through each diamond 
can be expressed as Xi + a ;2 + — 2 * ccq 


in Table we not only obtain more accurate predictions of 
running time, we can also give error bounds for our estimates. 

III. Algorithm 

A. Overview 

We now give an overview of our algorithm. Recall, that the 
problem studied can be considered as follows; Given a DAG 
with source s and sink t, find the longest source-to-sink path 
where the lengths of that paths are modeled as described in 
the Preliminaries, Section [HI 

The algorithm in im expresses every path as a linear 
combination of basis paths; using their lengths to estimate 
the length of the paths not measured. Intuitively, if two paths 
overlap (they share common edges) then knowing the length of 
one provides some information about the length of the other. 
Even basis paths with zero coefficient in the linear combination 
can provide information about the length of an estimated path. 

In our algorithm, we write integer linear programs (ILPs), 
with one constraint per measurement, looking for the longest 
path with the edge weights consistent with the measurements 
and Hx- Even though, are not observable, we show how to 
obtain consistent bound on firaax from the measurements. 

B. Path Extraction 

In this section we assume that we have a set of measure¬ 
ments AA, consisting of pairs (cc, lx) where a; is a path and lx is 
the measured length of x and we show how to find the longest 
path consistent with the measurements. In Section |III-D| we 
then show how to actually calculate the set S. To make the 
notation consistent with im, we call the measured paths the 
basis paths, even if they do not necessarily form a basis in 
the underlying vector space as was the case in uni. 

Suppose, for the moment, that the value of Pmax is known 
and equal to D G M.. Then the following problem encodes 
the existence of individual edge weights (we) such that the 
cumulative sum ij^eex along each measured path is 
consistent with its measured length; that is, the measured value 
differs by at most D from the cumulative sum. 


Problem 1. Input: DAG G, a set of measurements AA and 
D G R 

max \eii(path) 

S.t. k - D < J2eexi We <li+ D 

for each measurement {xi,li) G AA 
vars : We> 0 for each edge e 

Where maxlen(pof/i) expresses the length of the longest 
cumulative sum along some source-to-sink path in the graph. 
We now turn this problem into an ILP by expressing the 
existence of a path as follows: 

Problem 2. Input: DAG G, a set of measurements AA and 
D G R 

inax Y.eeEP<^ 

s.t. k - D <Y,eexi'^e < k + D 

for each measurement (xi, If) G AA 

'^eeout(s) ~ 1 

^^eGin{t) 

for each vertex v ^ {s, f} 

Pe < We 

for each edge e 

Pe < M -be 

for each edge e 
vars : for each edge e: 

We > 0 

be G {0,1} 

Pe>0 

Where M S K is a constant larger than any potential We. 
In the implementation, we take M to be the largest in the 
set of measurements S plus one. 

In the above ILP (Problem]^, Boolean variables be’s specify 
which edges of the graph are present in the extremal source-to- 
sink path and pe’s shall equal be x We. Thus, 'Yfe&EPe denotes 
the length of the extremal source-to-sink path. 

The existence of a path is encoded by the constraints 
specifying that there is a flow from the source to the sink. That 
is, that exactly one edge from the source has be = 1, exactly 
one edge to the sink has be = 1 and that for all intermediate 
vertices, the number of incoming edges to that vertex with 
be = 1 equals the number of outgoing edges from that vertex 
with be = 1. 

Eurther, for each edge e G E, we use the variable Pe to 
denote the multiplication pe = be ■ We. As be G {0,1}, we 
have Pe < We. Also, the constraints Pe ^ Ad ■ Pe ensure that 
if 6e = 0 then pe = 0. On the other hand, if = 1 then 
constraints imply only that 0 < Pe < We. Einally, note that the 
objective function is to maximize Jfe^EPe- hence, if 6e = 1 
then optimum value for Pe is to set pe to We. Hence, in the 
optimal solution, if = 1 then pe = We = I ■ We = be ■ We 
as desired. 

Recall, that for measurement {xiflf) it holds that f = 
Jfes^Xi We + d where \d\ < px < Pmax- Thus, in general. 




D needs to be at least 0 to ensure that Problem |2] is feasible. 
By the assumption, taking D = /Tmax yields a feasible ILP. 

Lemma 1. Problem is feasible for D = /Xmax- 

However, the value of /imax is neither directly observable 
nor known as it depends on the actual hardware the program 
is running on. We now show how to obtain a valid D yielding 
a feasible ILP in Problem |2] Later we show under what 
circumstances the solution of the resulting ILP gives the 
correct longest path. 

Consider the following Llfl 

Problem 3. Input: DAG G and a set of measurements M 


mm 

T 

s.t. 

f /^ — ^^eCiXi d- p 


for each measurement (xi, If) G AA 

vars : 

for each edge e: 


We > 0 


p > 0 


Intuitively, the problem above finds the least value of D for 
which Problem]^ is feasible, i.e., the least D consistent with 
the given set of measurements M. Formally, we have: 

Theorem 2. Let p{p) be the solution of Problem Then 
taking D = p{p) in Problem^yields a feasible ILP. 

Proof: First, note that Problem always has a solution, 
e.g., take We = 0 and p, = max^ li. 

Notice that, assuming there is at least one source-to-sink 
path, the only possible way for Problem to be infeasible 
is that D is small enough so that the constraints h — D < 
SeGx We < h + D are inconsistent. 

However, by the construction of Problem p{p) satisfies, 
k - P{p) < J2e&xi We < h + p{p) for every path Xi. The 
result now immediately follows. ■ 

Note that, by construction, taking p = /imax in Problemj^is 
feasible. Hence, D < /tmax as D is the least value consistent 
with the measurements. 

Notice that the solution of Problem can be used as a 
measure of timing repeatability of the underlying hardware 
platform. In case of perfect timing repeatability, that is if each 
edge (each statement of the underlying program) always took 
exactly the same time (regardless of concrete values of cache 
hits, misses, branch predictions, etc) to execute and that = 
0 for every measured path, then the solution of Problem 
would be 0. Conversely, the larger the value of the solution 
of Problem the bigger the discrepancy between different 
measurements. 

To measure the effect of timing repeatability on the com¬ 
puted value of D, we have taken measurements for a set of 
benchmarks used to evaluate our tool on (Section |IV-B| ) and 
randomly perturbed the measured execution times. We have 
perturbed each measurement randomly by up to 10%, 25% and 
50%. Table [I] shows the calculated values of D. As expected, 
the larger the perturbation, the larger the calculated value of D. 

^Note that Problem 3 is a linear program and not an integer linear program. 


TABLE I: Different values of D obtained in the benchmarks 
by perturbing the measurements. 


Benchmark 

Perturbation 

0% 

10% 

25% 

50% 

altitude 

57.0 

66.1 

87.8 

126.5 

stabilisation 

343.2 

371.3 

807.1 

1107.2 

automotive 

1116.0 

1281.3 

1486.9 

2961.3 

cctask 

73.9 

110.6 

150.9 

270.6 

irobot 

37.2 

95.9 

288.8 

552.8 

sm 

0.1 

23.2 

117.4 

216.8 


C. Optimality 

The solution of Problem[^is the best estimate of the longest 
path that is consistent with measurements AA. We now show 
how good the estimate is. 

Consider the solution of Problem with D equal to the 
solution of Problem]^ For each edge e, let p{we) denote the 
value of the variable We in the solution, and let r be the path 
corresponding to the solution of Problem]^ Denote the length 
of T in the solution by p(len(r)). We now show how much 
p(len(r)) differs from the actual length of r. Specifically, we 
shall show that the goodness of the estimate of the length of 
T is related to the following ILlfl 

Problem 4. Input: DAG G and a set of measurements AA 

max I \en{path) \ 

S.t. -1 < < +1 

for each measurement (xi^li) S AA 
vars : We for each edge e 

The existence of a path and the length of the path is 
expressed in the above ILP in exactly the same way as was 
done in Problem]^ Note that the above ILP is always feasible 
with I \en{path) at least 1; one solution is to set We = I for 
one edge outgoing from the sink and set We = 0 for all other 
edges. Further, note that Problem [^depends only on the graph 
and the set of the measured basis paths; it is independent of 
the (measured) lengths of the paths. In fact, we can show that 
as long as some path does not appear in the measurements 
AA, the solution of the above ILP is strictly greater than 1. 

Theorem 3. Let G be a DAG, AA a set of measurements and 
TT a source-to-sink path in G such that tt is not present in AA. 
Then the solution of Problem is strictly greater than 1. 

Proof: We give a satisfying assignment to variables We 
in Problem 1^ such that len(7r) > 1. 

Specifically, let Ci be the first edge of tt, that is, the edge 
outgoing from the sink of G. Further, let, D = {{u,v) \u G ir} 
be the set of edges with the initial vertex lying on tt. Then the 
assignment to weights We is as follows: 

e = Ci 
eG D 
otherwise 

'^To find the absolute value |len(pat/i)| we solve two linear programs. 
One with the objective function max len(paf/i) and one with the objective 
function max — len(path). 



















Note that, with this assignment to We’s, the length of pi 
equals len(7r) = 1 + > 1. Now, consider any other path t 

used in measurements Ai. In particular, r tt. There are |ii^| 
edges in G and the weight We associated with each edge e is 
at least —j^- Hence, len(T) > \E\ x = —1- 

Now, if r does not include e^, that is ^ r then, len(T) < 0 
as Wf-i is the only positive weight. If t includes e^, that is 
Bi G T, then r necessarilly contains at least one edge from 
D as T is different from tt. Hence, len(T) < 1. In any case, 
— 1 < len(r) < 1 as required and thus we have given a valid 
assignment to We’& with len(7r) >1. ■ 

Recall that the set S denotes the set of paths occurring 
measurements Ai. Let r{we) and r{pLx) be the real values of 
We for each edge e and for each path x G S. Then for each 
edge e the expression \p{we) — T(we)\ denotes the difference 
between the calculated values of We and the actual value of 
We- Analogously, the expression extends to entire paths; for 
a path X we have p{wx) = X]eGxP(tfe)' Now, the difference 
for the worst path can be bounded as follows. 

Theorem 4. Let k be the solution of Problem Then 

\p[len{T)) - r{len{T))\ < 

Proof: Note that by construction, r{we) and r{px) are a 
solution of Problem 1^ Hence, for every {xi,li) € Ai it holds 
that. 


eGxi 

Recall that D < p^ax and that that p{we) is a solution of 
Problem 1^ Hence, for every {xi,li) G Ai it holds that; 




< k - D < ^ p{We) < k + D < k 




e^Xi 


Hence, by subtracting the last two equations from each 
other, we have for any basis path Xi G S that; 

^-praax — E p{We) - r{We) < 2p max 

eGxi 

Now, dividing by 2pxaax we have; 

Y.e&x. P{We)-r{We) 


-1 < 


2/in 


< 1 . 


for any basis path Xi G S. 

Thus, the above inequality implies that taking 

P2{We) - r{We) 


We = 


2/in 


is a (not necessarily optimal) solution of Problem Since k 
is the length of the longest path achievable in Problem it 
follows that for any path x (not necessarily in the basis), we 
have 

SegxP(^e) -r(We) 

2/imax 


TABLE II; Comparison of the accuracy in the longest path 
extraction between our algorithm and the one in ifTH . For our 
accuracy, we take 2*k where k is the solution of Problem 
For im we take 2 * {# basis paths). 


Benchmark 

# Basis Paths 

il 111 Accuracy 

Our Accuracy 

altitude 

6 

12 

10.0 

stabilisation 

10 

20 

16.4 

automotive 

13 

26 

14.0 

cctask 

18 

36 

34.0 

irobot 

21 

42 

18.0 

sm 

69 

138 

48.6 


By rearranging, we have 


-2fc/iniax < '^P{We) “ r{We) < 2kp max- 
e^x 

In other words, the calculated length differs from the real 
length by at most 2fc/iniax, as desired. ■ 

Recall that the algorithm in O has difference between the 
estimated and the actual length at most 2\E\p^a.^ whereas our 
algorithm has 2fc/imax where k is the solution of Problem 
Observe that the dependence in the error term on p^ax is 
unavoidable as Pmax is inherent to the timing properties of 
the underlying platform. 

For comparison, we have generated the same basis as in ca 
and calculated the corresponding fc’s for several benchmarks. 
Table summarizes the results (see Table IV for the descrip¬ 
tion of benchmarks). As can be seen from the table, when 
using the same set of measurements, our method gives more 
accurate estimates than the one in ifTTl . 

Furthermore, recall that in Problem we calculate the 
best (lower) bound D on Pmax consistent with the given 
measurements. Together with the above theorem, this gives 
“error bounds” to the estimate in Problem Specifically, if 
the length of the longest path computed in Problem is T 
then, the length of the path when measured, that is consistent 
with the measurements is within T ± {2k x D). However, 
note that this is only the best bound deducible from the 
measurements since D < p^ax- Since /j-max is not directly 
observable and we assume no nontrivial bound on Pmax, the 
length of the path cannot be bounded more accurately without 
actually measuring the path. 


The above analysis applies to the extraction of the single 
longest path. Now, suppose that instead of extracting just one 
longest path, we want to extract K longest paths. To that 
end, we iterate the above procedure and whenever a path is 
extracted, we add a constraint eliminating the path from the 
solution of Problem and then solve the updated ILP. For a 
path X, the constraint eliminating it from the solution space of 
Problem < |a;|- The constraint specifies that not 

all the edges along x can be taken together in the solution. 

As the length of the predicted and the measured length 
differ, it may happen (e.g.. Table that when measured 
the length of the path predicted to be the longest is not 
actually the longest. Thus, to find the longest path, we may 





















need to iterate the above process by generating paths with 
ever smaller predicted lengths, stopping whenever the current 
estimate differs by more than (2k x D) from the longest 
estimate. 

D. Basis Computation 

The algorithm (Problem to estimate the longest path 
depends on the set of measurements M of the basis paths S. 
In this section we show how to calculate such a set of paths. 
In general, arbitrary set of paths can be used as basis paths. 
For example, we have shown in Table [III] that using the set of 
paths used in CD, we are able to get more accurate estimates 
than those obtained in CD- 

Recall that (Theorem the accuracy of the solution of 
Problem is tightly coupled with the solution of Problem 
This leads to a tunable algorithm, which depending on the 
desired accuracy of the predictions, calculates a set of paths 
to be used in Problem 

Specihcally, given a desired accuracy A S K, A > 1, we 
want to find a set of feasible paths S such that the solution of 
Problem is at most A. We implemented a simple iterative 
algorithm (Algorithm [^l that finds such a set by repeatedly 
extending the set of paths by the path corresponding to the 
solution of Problem 0] 

In the algorithm, if the longest extracted path is infeasible 
in the underlying program, we add a constraint into the ILP 
(Problem]^ prohibiting the path. That is, if the longest path is 
infeasible and r is the unsatisfiable core of the longest patI0 
then we add a constraint that not all the corresponding to 
the edges used in r are set to 1 in Problemi.e., X^eGr ^ 
|r| where |t| denotes the number of edges in r. Then we solve 
the updated ILP. 

S'^ 0 

while (Solution of Problem^with paths S') > A do 
X ^ longest path in Problem 
if X is feasible then 
I S^SU{x} 

else 

I Add a constraint prohibiting x 

end 

end 

return S 

Algorithm 1: Iterative algorithm for basis computation 

Theorem 5. If A> 1 then the Algorithm terminates with a 
set P of paths such that the solution of Problem with paths 
P is at most A. 

Proof: Note that each constraint in Problem limits the 
length of some path to (at most) 1. In particular, if S contains 
all the paths in the graph then the solution of Problem 
equals 1. 

^Minimal set of edges that cannot be taken together as identified by an 
SMT solver 


TABLE III: Number of paths generated by Algorithm to 
reach the desired accuracy (second column). Third column 
shows the accuracy (solution of Problem of the generated 
set of paths 


Benchmai'k 

Desired k 

Actual k 

# Basis Paths 

Time(s) 


10 

5.0 

7 

0.03 

altitude 

5 

5.0 

7 

0.03 


2 

1.0 

10 

1.66 


10 

7.0 

11 

0.10 

stabilisation 

5 

4.7 

12 

0.96 


2 

2.0 

40 

22.78 


10 

7.0 

14 

0.14 

automotive 

5 

5.0 

14 

0.89 


2 

2.0 

30 

27.60 


10 

9.0 

19 

0.20 

cctask 

5 

5.0 

25 

4.10 


2 

2.0 

76 

42.91 


10 

9.0 

22 

0.50 

irobot 

5 

5.0 

34 

20.13 


2 

2.0 

118 

182.42 


22 

21.8 

70 

328.14 

sm 

18 

18.0 

73 

7089.04 


15 

14.5 

77 

10311.49 


Further, if the algorithm hnds some path x to be the longest 
in some iteration then the length of x in all subsequent 
iterations will be at most 1 as a; G S'. Therefore, as long as the 
solution is greater than 1, the longest path found is different 
from all the paths found in the previous iterations. 

Also, if the path is infeasible, then a constraint is added 
that prevents the path occurring in any subsequent iterations. 
It follows from these considerations that the algorithm keeps 
adding new paths in each iteration and eventually terminates. 

By construction, the solution of Problem with the set of 
paths S is at most A. ■ 

In the extreme case of A = 1, it immediately follows from 
Theorem that Algorithm [T] returns all feasible paths in the 
underlying graph. 

We have implemented the above iterative algorithm and 
evaluated it on several case studies. Table m summarizes the 
number of paths generated by the algorithm for the given 
accuracy k as well as the running time required to hnd those 
paths. 

We have observed that the basis computation took sub¬ 
stantial part of the entire algorithm. However, notice that the 
basis-computation algorithm (Algorithm 0 need not start with 
S' = 0 and works correctly for any initial collection of paths S. 
Therefore, as an optimization, we first compute the initial 
set of paths S using the original algorithm from [TD, which 
computes the 2-barycentric spanner of the underlying DAG. 
Only then we proceed with the iterative algorithm to find a 
set of paths with the desired accuracy. 

Figure shows the performance of the iterative algorithm 
on two benchmarks. The decreasing (blue) line shows how 
the accuracy k decreases with each path added to the basis. 
The increasing (red) line shows the time needed to hnd 
each path. The hgure shows only the performance after the 
precomputation of the 2-barycentric spanner. 
















Fig. 4; Performance of the Algorithm The decreasing (blue) 
line shows the length of x computed {k) in line 3. The 
increasing (red) line shows the time taken to perform each 
iteration. 

(a) cctask 



(b) irobot 



# paths 


IV. Evaluation 

A. Implementation 

The algorithm to identify the longest (up to accuracy k) 
path a given program P is shown in Algorithm 

We now briefly describe the implementation of main stages 
of the Algorithm]^ Our implementation is build on top of ifTTl . 
See im for further discussion of technical details. 

CFG extraction The procedure begins by building the 
CFG G of the given program. The vertices correspond to 
locations and edges to individual statements. The extraction 
is build on top of CIL front end for C |[T4l . Note that all ILPs 
(Problems 1^ and 1^ introduce a variable per each edge of G 


Extract CFG G from the program P 
S ^ basis with accuracy at most k (Algorithm 
D ^ Solution of Problem 0 
T ^ Solution of Problem with paths S and D 
return r and its estimated length 
Algorithm 2: Algorithm to And the worst-case execution 
time. Input: Program P, accuracy k 


and that each problem optimizes for the length of a source-to- 
sink path. Thus, if some edge is always followed by another 
one (the in- and out-degree of the joining vertex is one) then 
the edges can be merged into a single on^ without changing 
the solution of the ILP problems. Therefore, we process G 
by merging edges that lie on a single path into a single edge. 
This reduces the number of variables used and improves the 
performance. 

Basis computation The basis is computed as described in 
Section III-D[ Algorithm [T] We use Gurobi solver ifTSl to 
solve individual ILPs and use the 2-barycentric spanner as 
the initial set S. Solving the ILPs posed the main bottleneck 
of our approach. 

Input generation Each path through the CFG G corre¬ 
sponds to a sequence of operations in the program and hence 
a conjunction of statements. For a given path, we use the 
Z3 SMT solver GH to And an input that corresponds to 
the given path or to prove that the path is infeasible and no 
corresponding input exists. In the experiments, SMT solving 
was fairly fast. 

Longest-path extraction We solve Problem [fusing Gurobi 
ILP solver ISl. If the extracted path tt is infeasible, we add 
a constraint (X^eGir ^ eliminating the path from the 
solution and solve the new problem. Similarly, we solve for 
K longest paths; we add a constraint prohibiting the extracted 
path and solve the resulting ILP again, repeating until we 
successfully generate K feasible paths. 

Recall that the calculated length of the path is accurate only 
up to the precision 2k x D. Thus, we can repeat the process 
until the length of the extracted path is outside of the range 
for the longest extracted path. However, in practice we did 
not And this necessary and found the longest path after a few 
iterations. 

Path-Length Measurement To measure the execution time 
of the given program on a given input, we create a C program 
where we set the input to the given values and then run it using 
a cycle-accurate simulator of the PTARM processor ini. 


B. Benchmarks 

We have evaluated our algorithm on several benchmarks 
and compared it with the algorithm in ifTTIl . We used the same 
benchmarks as in ED as well as benchmarks from control 
tasks from robotic and automotive settings. The benchmarks 
in ED come from Malardalen benchmark suite ESI and the 

^For example, in Figure M every diamond can be replaced by two edges, 
one edge for the top half and one edge for the bottom half. 

































TABLE IV: Number of nodes, edges and paths in the CFGs 
extracted from the benchmarks 


Benchmark 

# Nodes 

# Edges 

# Paths 

altitude 

36 

40 

11 

stabilisation 

64 

72 

216 

automotive 

88 

100 

506 

cctask 

102 

118 

657 

irobot 

170 

195 

8136 

sm 

452 

523 

33,755,520 


PapaBench suite ||T9| . The authors of ifTTIl chose implementa¬ 
tions of actual realtime systems (as opposed to hand-crafted 
ones) that have several paths, were of various sizes, but do not 
require automatic estimation of loop bounds. 

Since we assume all programs contain only bounded loops 
and no recursion, we preprocessed the programs by unrolling 
the loops and inlining the functions where necessary. Table |IV| 
summarizes properties of the benchmarks used. 

Table |V] shows the lengths of five longest paths as found by 
our algorithm and the one in lim . The first half of the table 
shows the results as obtained by the algorithm in ifTTl . The 
second half shows the results as obtained (together with the 
“etTor bounds”) by our algorithm (Algorithm]^. 

Note that in each case the longest path returned by our 
algorithm is never shorter than the longest path found by in. 
In half of the cases, our algorithm is able to find a path longer 
than the one in in. Also notice that the actual measured 
length is always within the computed “etTor bounds”. 

The biggest benchmark, sm, is a collection of nested switch- 
case logic setting state variables but with minimal compu¬ 
tations otherwise. Hence, there is a large number of paths 
(33, 755,520) in the CFG yet, as expected, the computed D 
is small. 

V. Conclusion 

In this paper, we have addressed the problem of estimating 
the worst-case timing of a program via systematic testing 
on the target platform. Our approach not only generates an 
estimate of worst-case timing, but can also produces test cases 
showing how that timing is exhibited on the platform. Our 
approach improves the accuracy of the previously published 
GameTime approach, while also providing error bounds on the 
estimate. 

Note that our approach can be adapted to produce timing 
estimates along arbitrary program paths. In order to do this, 
one can fix variables be in Problem suitably. Thus, we can 
also estimate the longest execution of a given path that is 
consistent with the measurements. 

In the paper we have analyzed the timing behavior of a 
given program. However, instead of measuring cycles we can 
measure energy consumption of the program executions. The 
same techniques can then be applied to find the input consum¬ 
ing the most energy. In general, the approach presented in this 
paper can also be extended to other quantitative properties of 
the program and is not limited only to the WCETT analysis. 
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TABLE V; Comparison of the results produced by the algorithm presented in this paper and in ca for generating the top five 
longest paths for a set of benchmarks. Column Predicted shows the length predicted by each algorithm. Column Measured 
show the running time measured when run on the corresponding input. For our paper, we give “error bounds” of the form 
k X D. The column Time shows the time it takes to generate the estimates and test cases. The largest measured value per each 
benchmark is shown in bold. 


Benchmark 

GameTime fll 


Our Algorithm 

Predicted 

Measured 

Time(s) 

Predicted 

Measured 

Time(s) 

altitude 

867 

867 

11.7 

909 ± 1.0 X 57.0 

867 

14.4 

789 

789 

815 ± 1.0 X 57.0 

758 

776 

751 

732 ± 1.0 X 57.0 

789 

659 

763 

719 ± 1.0 X 57.0 

737 

581 

581 

638 ± 1.0 X 57.0 

581 

stabilisation 

4387 

3697 

26.4 

4303 ± 2.0 X 343.0 

3599 

77.2 

4293 

4036 

4302 ± 2.0 X 343.0 

4046 

4290 

3516 

4285 ± 2.0 X 343.0 

3944 

4286 

3242 

4284 ± 2.0 X 343.0 

3516 

4196 

3612 

4248 ± 2.0 X 343.0 

3697 

automotive 

13595 

8106 

47.0 

11824 ±2.0 X 1116.0 

10982 

93.8 

11614 

9902 

11696 ±2.0 X 1116.0 

10657 

11515 

11515 

11424 ±2.0 X 1116.0 

10577 

11361 

5010 

11338 ±2.0 X 1116.0 

11515 

11243 

11138 

9830 ±2.0 X 1116.0 

9263 

cctask 

991 

808 

29.4 

870 ± 2.0 X 73.0 

861 

138.4 

972 

605 

869 ± 2.0 X 73.0 

858 

943 

852 

866 ± 2.0 X 73.0 

897 

936 

848 

865 ± 2.0 X 73.0 

897 

924 

821 

861 ± 2.0 X 73.0 

873 

irobot 

1462 

1430 

50.9 

1451 ± 2.0 X 37.0 

1406 

269.0 

1459 

1463 

1450 ± 2.0 X 37.0 

1411 

1457 

1418 

1449 ± 2.0 X 37.0 

1411 

1454 

1451 

1448 ± 2.0 X 37.0 

1411 

1454 

1463 

1448 ± 2.0 X 37.0 

1464 


2553 

2550 


2552 ± 21.81 X 0.2 

2550 



2551 

2550 


2551 ± 21.81 X 0.2 

2550 


sm 

2536 

2537 

211.0 

2536 ± 21.81 X 0.2 

2537 

3290.2 


2534 

2537 


2536 ± 21.81 X 0.2 

2537 



2532 

2537 


2531 ± 21.81 X 0.2 

2537 







































































